Economic evaluation of participatory women’s groups scaled up by the public health system to improve birth outcomes in Jharkhand, eastern India

An estimated 2.4 million newborn infants died in 2020, 80% of them in sub-Saharan Africa and South Asia. To achieve the Sustainable Development Target for neonatal mortality reduction, countries with high mortality need to implement evidence-based, cost-effective interventions at scale. Our study aimed to estimate the cost, cost-effectiveness, and benefit-cost ratio of a participatory women’s groups intervention scaled up by the public health system in Jharkhand, eastern India. The intervention was evaluated through a pragmatic cluster non-randomised controlled trial in six districts. We estimated the cost of the intervention at scale from a provider perspective, with a 42-month time horizon for 20 districts. We estimated costs using a combination of top-down and bottom-up approaches. All costs were adjusted for inflation, discounted at 3% per year, and converted to 2020 International Dollars (INT$). Incremental cost-effectiveness ratios (ICERs) were estimated using extrapolated effect sizes for the impact of the intervention in 20 districts, in terms of cost per neonatal deaths averted and cost per life year saved. We assessed the impact of uncertainty on results through one-way and probabilistic sensitivity analyses. We also estimated benefit-cost ratio using a benefit transfer approach. Total intervention costs for 20 districts were INT$ 15,017,396. The intervention covered an estimated 1.6 million livebirths across 20 districts, translating to INT$ 9.4 per livebirth covered. ICERs were estimated at INT$ 1,272 per neonatal death averted or INT$ 41 per life year saved. Net benefit estimates ranged from INT$ 1,046 million to INT$ 3,254 million, and benefit-cost ratios from 71 to 218. Our study suggests that participatory women’s groups scaled up by the Indian public health system were highly cost-effective in improving neonatal survival and had a very favourable return on investment. The intervention can be scaled up in similar settings within India and other countries.


Introduction
There has been a significant decline in the number of neonatal deaths globally, from 5 million in 1990 to 2.4 million in 2020. However, the reduction in deaths during the neonatal period has been slower than that observed for the post-neonatal period [1]. In addition, 80% of neonatal deaths occur in sub-Saharan Africa and South Asia, with neonatal mortality rates (NMR) of 27 and 25 deaths per 1,000 live births, respectively, in 2020 [1]. To achieve the global Sustainable Development Target of neonatal mortality of no more than 12 per 1000 livebirths by 2030 [2], countries with high NMR need to implement evidence-based, cost-effective interventions at scale [3,4].
Community mobilisation through women's groups practising Participatory Learning and Action (PLA) is recommended by the World Health Organisation's (WHO) Global Strategy for Women's, Children's and Adolescents' Health [5] as a cost-effective intervention to improve neonatal survival. This strategy engages communities in a series of monthly meetings following the four phases of PLA: identifying and prioritising health problems in the perinatal period, identifying feasible strategies to address these issues, implementing the strategies, and evaluating the process [6]. Participatory women's groups have been implemented in small-to medium-scale efficacy trials in Nepal, India, Bangladesh and Malawi [7][8][9][10][11], with several trials demonstrating significant cost-effective reductions in neonatal mortality [12][13][14].
To accelerate reduction in neonatal mortality in India, where around a fifth of all neonatal deaths happen each year [15], India's National Health Mission (NHM) in 2016 requested 10 states to consider scaling up participatory women's groups. This decision was informed by results from two previous efficacy trials in Jharkhand and Odisha [9,11], a meta-analysis of seven trials of participatory women's groups [16] and the WHO recommendation [6]. Between 2016 and 2020, with support from Jharkhand NHM, participatory women groups were scaled up across 24 districts of Jharkhand state, covering a population of around 30 million [17,18]. The programme was called FLAG (Facilitated Learning and Action Groups) and was delivered by public health system frontline government workers called Accredited Social Health Activists (ASHAs). The programme's effects were evaluated through a pragmatic cluster non-randomised controlled trial in six of Jharkhand's 24 districts. The trial found a 24% reduction in neonatal mortality, including 26% among the most deprived [19].
This study aimed to assess the cost and cost-effectiveness of the FLAG programme at scale.

Study setting and participants
Jharkhand is a state of Eastern India with a population of around 40 million (estimated for 2021), around 76% of which is rural. Indigenous communities (Scheduled Tribes) constitute 26% of the population [20]. The maternal mortality ratio and neonatal mortality rate in Jharkhand state are 76 per 100,000 livebirths and 33 per 1000 livebirths, respectively [21,22]. All women living in the programme districts were eligible to take part in participatory women's groups run under the FLAG programme. The programme's impact evaluation included women of reproductive age (15-

FLAG intervention
The FLAG programme was implemented by the National Health Mission (NHM), Jharkhand, in partnership with Ekjut, a civil society organisation working with women's groups in Jharkhand since 2006. The programme began in six blocks of six districts in 2015-6, and gradually expanded to 21 of Jharkhand's 24 districts from 2017 onwards. The three remaining districts did not receive the intervention until 2019 and were 'comparison' districts for the programme's impact evaluation. A detailed description of the women's group intervention is reported elsewhere [19]. In brief, the intervention comprised monthly women's group meetings (around 36 monthly meetings) following a Participatory Learning and Action (PLA) cycle. Meetings were facilitated by ASHAs and their supervisors, who supervised approximately 15-20 ASHAs. The women's groups PLA intervention followed three phases. In the first phase, group members identified and prioritised maternal and newborn health problems in their community. In the second phase, they discussed the causes of problems prioritised in the first phase and identified locally feasible strategies to address the causes. During this phase, groups organised a larger community meeting to share their prioritised health problems and the strategies they had identified with the wider community. In the third phase, groups implemented their strategies. In the fourth phase, groups evaluated the overall cycle. In each phase, different participatory techniques such as voting to prioritise problems, storytelling and games were used to facilitate discussion.
Women's groups mainly focused on health problems in the perinatal period, but a number of extra meetings were added to phase three, focusing on issues such as infant and young child feeding, maternal nutrition and family planning. ASHAs, in addition to leading monthly PLA meetings, were also required to carry out their routine work in the community, including promoting antenatal care and institutional delivery, and providing postnatal home visits as part of the Home-Based Newborn Care programme [23,24].
With support from Ekjut, NHM District and Block-level staff gave ASHA Facilitators three rounds of training on PLA, with each training lasting five days. ASHA Facilitators then provided on-the-job training to all ASHAs in their catchment area (around 20 villages and 20 ASHAs), using an odd-even approach covering 10 villages each month. We described this lowcost training method in detail elsewhere [19]. ASHA Facilitators received an incentive of INR 1,000 (USD 13) to conduct 10 meetings per month, and ASHAs received INR 100 (USD 1.3) per meeting. ASHAs and ASHA Facilitators formed around one group per 1000 population and encouraged pregnant women and women from underserved groups such as Scheduled Tribes, Scheduled Castes and remote hamlets, to participate in group meetings. ASHA Facilitators and ASHAs also attended monthly review meetings with district-level coordinators as well as biannual meetings at the state-level to supplement their training.

Impact evaluation
We used a pragmatic cluster non-randomised controlled trial design to evaluate the effects of FLAG. Three of the six evaluation districts (Ranchi, West Singhbhum, Khunti) started women's groups in 2017, while the other three (Bokaro, Palamu, Dumka) started in 2019. The six evaluation districts were purposively selected and allocated to intervention or comparison in discussion with NHM Jharkhand, bearing in mind the feasibility of early intervention and ensuring that intervention and comparison districts had a similar rural population size. Within the six evaluation districts, Ekjut, NHM Jharkhand and University College London then selected 20 blocks (subdistrict administrative units of c. 100 000 population) for data collection by matching 10 blocks in the early arm to 10 blocks in the delayed arm. Matching was done on the basis of % of Scheduled Tribe and Scheduled Caste population, the Infant Mortality Rate, and % of women delivering in health facilities. In each block, the evaluation team further purposively selected five data collection units created to cover around 10 000 population each, or a total of 100 units. The evaluation team collected data on births, deaths, and perinatal practices among women of reproductive age prospectively in these 100 purposively selected geographic units covering an estimated total population of 1 038 718.
We collected data during a six-month baseline period (1 st March-31 st August 2017) and a 24-month evaluation period (1 st September 2017-31 st August 2019). The trial's primary outcome was neonatal mortality. Secondary outcomes included stillbirths, perinatal mortality, maternal mortality, as well as preventive and care-seeking practices for women during the perinatal period [19]. We used an intention-to-treat approach and included data from all eligible mothers in the evaluation area. All trial analyses were carried out a cluster-level, with districts as clusters. All models were adjusted for tribe/caste, maternal schooling, maternal literacy, asset quintile, and baseline values for each outcome. More details on the evaluation design and analysis are provided elsewhere [19]. The trial found a 24% reduction in neonatal mortality (Adjusted Odds Ratio [AOR]: 0.76, 95% CI: 0.59-0.98) in districts that began the intervention in 2017 ('early districts') compared to those than began in 2019 ('delayed districts') [19].

Impact at scale and life years saved
We extrapolated the effect of the intervention on neonatal mortality found in the six trial districts to other districts in the State that met three conditions: received all PLA trainings; completed two-thirds or more of perinatal meetings by 2019 and; had 30% or more of pregnant women in ASHA catchment areas participating in group meetings. A Monitoring and Information System was established by Ekjut and NHM to track trainings and meetings throughout the State. In addition, intervention quality was monitored by an independent organisation in the 21 districts that started the intervention in 2017. In discussion with NHM and Ekjut, the 30% coverage criterion was met if more than 30% of pregnant women participated in at least 9 out of 19 meetings observed in each district every three months. Overall, twenty out of 24 districts met these criteria. We estimated the number of neonatal lives saved in rural areas of 20 districts using the following formula:

Pr * Br * NMR * ð1À RRÞ
where Pr is rural population in 20 districts, Br is crude birth rate in rural area, NMR is neonatal mortality rate in rural areas without intervention, and RR is the Risk Ratio for the intervention in the evaluation districts [19]. We then estimated life years saved by multiplying the number of neonatal deaths averted in 20 districts by 30.81, which corresponds to a standard life expectancy of 86 years at birth (representing life expectancy of Japanese females in 2010) recommended by Global Burden of Diseases 2010 [25], and discounted at 3%.

Intervention costing
We estimated the cost of the intervention implemented in 20 districts from a programme provider perspective. The costing's time horizon was 42 months (October 2016-March 2020). A combination of activity-based, expenditure and ingredient approaches [26] were used to estimate the costs. We collated financial costs data retrospectively from the expenditure reports and project accounts of implementing agencies, namely Ekjut and NHM Jharkhand. In addition, we used project records on the number of meetings held by ASHAs and incentive data (i.e. 100 INR per meeting) to estimate the total incentives paid to ASHAs. Cost data were input into an excel-based tool and categorised according to line item, intervention activities, and implementing agency (Ekjut and NHM). Table 1 provides a description of cost categories. Staff costs were allocated to different activities using staff time-use data collected through consultation with the project staff. We excluded all research costs as these were primarily incurred by the impact evaluation.

Analyses
All costs were adjusted for inflation, discounted at 3% per year as recommended by WHO-CHOICE [27] and the Gates/iDSI Reference Case for Economic Evaluation [28], then converted to 2020 international dollars (INT$) using the Purchasing Power Parity conversion factor of 21.2 for India [29]. Incremental cost effectiveness ratios (ICERs) were estimated using the outcome data from the effect at scale analysis and presented in terms of cost per neonatal death averted and cost per life year saved. We have followed recommendations from the Gates/iDSI Reference Case for Economic Evaluation [28] and Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statemen [30] in our analysis and reporting (refer to S1 Checklist).
In addition to the ICER, we also estimated unit costs for the intervention in terms of total and average annual costs per livebirth covered by the intervention. Sensitivity analysis. We conducted a series of deterministic one-way sensitivity analyses and probabilistic sensitivity analyses (PSA) using Monte Carlo simulation. We quantified the impact of the following parameters in one-way sensitivity analyses: intervention effectiveness (95% confidence interval), intervention cost (+/-25%), discount rate (differential rate for cost and outcome -0%-10%), life expectancy (standard 86 years vs local life expectancy and WHO Global Health Estimates), baseline neonatal mortality rate (NMR), and rural crude birth rate.
In the PSA, we tested simultaneous variations of multiple parameters, using their assumed distribution, generating a total of 1000 iterations in Excel. We created cost-effectiveness Benefit cost analysis/ return on investment. In ideal circumstances, a context-specific estimate for the value of premature mortality averted is always preferred to a generic estimate. Some Indian studies have estimated the value of statistical life (VSL) based on a stated preference [32] or as part of a study on wage compensation [33,34], but values reported by these studies were widely different. We therefore estimated the monetary value of avoided neonatal deaths following the benefit transfer approach suggested by Robinson et al [35]. Using this approach, we estimated the value of VSL and value of a statistical life year (VSLY). VSL reflects an individual's willingness to pay for a change in his/her own risk of mortality (or life expectancy in case of VSLY) within a specific time period (usually a year) [35], and is not an indication of an individual's or society's valuation of a human life [36].
Robinson et al [35] recommended following three alternative approaches to estimate VSL: (1) extrapolating a VSL from the US (US$ 9.4 million in 2016, equivalent to 160 times Gross National Income-GNI per capita) with an income elasticity of 1.5; (2) estimating VSL as 160 times GNI per capita, with an income elasticity of 1.0 (US base value); (3) estimating VSL as 100 times GNI per capita, with an income elasticity of 1.0 (OECD base value). Income elasticity represents association between VSL and income. An income elasticity of greater than one suggests that the ratio of VSL to GNI per capita is smaller in lower income compared to higher-income populations [35]. We used GNI per capita of 2015 INT$ 6060 (or US$ 1600) for India in our calculations.
We also estimated a constant VSLY as recommended by Robinson et al [35]. VSLY is estimated by dividing the VSL by the undiscounted life expectancy of an adult of average age. We  [37]. We then multiplied the VSLY by the total neonatal life years saved to estimate total mortality benefits.
To be conservative, we used the VSLY calculated from approach (1), i.e. 160 times GNI per capita with an income elasticity of 1.5, with benefits discounted at 10%, as our base case scenario. We used discount rates of 3%, 5% and 10% (base-case) for benefit estimates as recommended by iDSI reference case [28] and Haacker et al [38]. We estimated net benefits by subtracting total intervention costs from total monetised mortality benefits, and the benefitcost ratio (BCR) by dividing total monetised benefits by total intervention cost.
Ethics. Ethical approval for the FLAG programme evaluation was obtained from an independent ethics research committee linked to Ekjut in Ranchi, and University College London's Research Ethics Committee (Reference: 1881/003). We sought individual informed consent from all participants in the evaluation study.

Results
The total and annual cost of programme implementation in 20 districts were INT$ 15,017,396 and INT$ 4,433,165, respectively. Intervention covered around 1.6 million livebirths across 20 districts which translated to INT$ 9.4 per livebirths (Table 2).
Staff costs constituted around 80% of all intervention costs. Around 40% of staff costs were incentives for ASHAs and ASHA facilitators. Mixed inputs (mainly trainings and meetings) were the second largest cost category, amounting to 13% of all costs (Table 3 and Fig 1). PLA implementation (which mainly include incentives paid to ASHAs and ASHA facilitators), and training constituted around 80% of all intervention costs. Preparation, planning and community sensitisation meetings and recruitment constituted less than 1% of total costs (Table 3 and Fig 2).
FLAG's impact evaluation found that the intervention saved 188 neonatal lives over a two year period (min: 16 -max: 327), using lower and higher bound of confidence interval around impact) in the three early intervention districts taking part in the evaluation [19]. Over the 42 months of scale up, we estimated that the intervention saved a total 11,803 neonatal lives across 20 districts (min: 1026-max: 20,527) [19]. This translates to an estimated 363,621 (discounted) life years saved, with a range of 31,609 to 632,386 (Table 4)

Sensitivity analysis
Fig 3 presents results from one-way deterministic sensitivity analyses. The results indicated that uncertainties around intervention effectiveness, NMR in areas without intervention, differential discount rates and rural crude birth rates led to significant variation in the ICER. For example, the uncertainty around intervention effectiveness (95% confidence interval risk ratio: 0.60-0.98), led to variations of -43% to 1050% in ICER, or INT$24 to INT$475 (S1 Table). Varying NMR rate in areas without intervention (from 18 to 52 per 1000 live births) led to variations of -38% to +76% in the ICER (i.e. INT$ 25-INT$73). In contrast, using local life expectancy of 70 or 91.9 used by WHO Global Health Estimates had only small effects on the cost effectiveness results (+5% and -1%, respectively). The 95% Confidence Interval around the ICER calculated from 1000 iterations in PSA was INT$32 to INT$ 62 (S1 Fig). Overall, varying uncertain parameters did not change the conclusion that the intervention is highly cost-effective. CEAC shows that at the lowest cost-effectiveness threshold of 1% of GDP per capita (INT$70), the probability of the intervention being cost-effective was 78% and at the highest threshold of 51% of GDP per capita (INT$3600), the probability was 99%. The probability of being cost-effective was higher than 90% for any thresholds above INT$150 (S2 Fig). Table 5 presents the results of the VSL, VSLY estimates. Table 6 shows total mortality benefits estimations. Estimated total benefits due to aversion of premature mortality, using US-VSLY extrapolation approach, ranged from INT$ 1,061 million with a 10% discount rate to INT$ 3,269 million with a 3% discount rate. The net benefit estimates ranged from INT$ 1,046 million (base case) to INT$ 3,254 million, and BCRs from 71 (base case) to 218.

Discussion
Our study is the first to assess the cost and cost-effectiveness of a large-scale community mobilisation intervention through participatory women's groups delivered by a public health system to improve neonatal survival. Our findings suggest that the scaled-up intervention was highly cost-effective in reducing neonatal mortality and has a very favourable return on  investment. The incremental cost-effectiveness ratio of $INT41 is substantially lower than India's GDP per capita (ranging from 0.3% to 7% of GDP). FLAG's ICER is 85% lower than that found in a smaller-scale efficacy trial of women's groups facilitated by ASHAs conducted in Jharkhand and Odisha (ICER of 2017 INT$ 274 or US$83), and 70% lower than in another smaller trial of women's groups supported by salaried facilitators in the same states (ICER of 2016 INT$ 135) [12,13]. Similarly, FLAG's ICER is also substantially lower than those reported for small scale efficacy trials of women's groups in Nepal, Bangladesh, and Malawi [12] (S2 Table). A similar conclusion can be drawn by comparing FLAG results with two small (JEEVi-KA-MC) [40] and large (Parivartan) scale [41] implementation studies of Self-help groups (SHGs) with maternal and child health components. The Parivartan programme [41] [40]. Although direct comparisons are impeded by different definitions of beneficiaries and reporting for different years, data suggest FLAG was more cost-effective than these two past programmes. A combination of factors might have contributed to the reduced costs of FLAG. Employing incentivised ASHA and ASHA facilitators rather than salaried facilitators and supervisors is potentially the main driver of cost savings. The 'odd-even' on-the-job-training approach used at scale is another potential driver. The intervention was implemented in a less intensive manner than in previous small scale efficacy in Jharkhand [9,13]. In FLAG, there was one women's group per c. 1000 population compared to one per c.500 population in previous trials in India. Finally, the results also potentially reflect economies of scale (See S3 Fig); the intervention area covered 1.6 million livebirths at a very low unit cost (cost per livebirths) (INT$ 9.4 compared with average unit cost of INT$ 203, range: 2016 INT$ 61-537) [12] in smaller scale trials of women's groups (S2 Table). As S3 Fig illustrates, unit costs decreased with the increased scale of the intervention, which is in line with conclusion from scale up of SHGs in India, in general [42].
The key drivers of cost-effectiveness, based on our sensitivity analysis, were intervention effectiveness, NMR in areas with no intervention, as well as the discount rate and rural crude birth rate. Sensitivity analyses showed that the intervention is still highly cost-effective in settings with NMR less than 32 deaths per 1000 livebirths, if all other parameters remain constant. This conclusion still stands if a lower NMR and the lower bound of the confidence interval for effectiveness is used (with ICER of INT$ 836). However, groups have not been implemented in rural settings with NMR of less than 20 deaths per 1000 livebirth in India or elsewhere [16], therefore the results for these settings should be interpreted with caution. Our cost-effectiveness results could be transferable to other districts in India or rural settings of other countries with high NMR and similar health system characteristics such as the accessibility of health services.  [43]. The BCR for FLAG is at the higher end of available BCRs reported for MNCH, alongside childhood nutrition interventions [43]. However, it should be considered that these studies are not directly comparable due to intervention type, methodology used and context of the studies, which mainly are from Sub-Saharan Africa.
Our study has three main limitations. First, we estimated costs from a program provider perspective, which is reflective of actual costs and potential costs of scaling up the intervention elsewhere in rural, underserved areas of India. We therefore did not include indirect costs/ opportunity costs to intervention participants, ASHAs and other community health providers affected by the intervention. Although it would be ideal to estimate full implementation costs from a societal perspective, it was not possible to collect more data due to the retrospective nature of the costing exercise. Based on our estimates of societal costs of a similar intervention in rural India [44], though at smaller scale, opportunity costs constitute around 35% of total costs. In addition, including these costs would not change the conclusion that the intervention is highly cost-effective. Second, using neonatal mortality as a primary outcome in our analysis likely underestimates the impact of women's groups. We have not included potential short and long-term health benefits beyond neonatal period and for mothers, children and their siblings. Observed improvements in exclusive breastfeeding, and postnatal visits might result in long-term health benefits for both mothers and children. Third, we did not fully explore the possibility of a 'rationing effect' whereby the introduction of PLA might cause ASHAs to invest less time in other mandated interventions. The impact evaluation found that the intervention increased postnatal ASHA home visits, signalling a positive synergy. However, we cannot fully rule out a rationing effect as we did not measure ASHAs' engagement in all mandated activities. Future research should explore whether and how ASHAs' other activities are supported or hindered by the introduction of PLA more systematically.

Conclusion
Our study indicates that participatory women's groups scaled up by the public health system in India were highly cost-effective in improving neonatal survival. The intervention can be scaled in similar settings within India and other countries.